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QUEUE HIERARCHY & QUOTAS 


YUNIKORN 
Hierarchical model 


- Resources are managed via queues 
- Auto created queues 
- Static queues 

- Resource located in leafs 
- Propagate up to the root 


- Requests (pending) 


- Allocation (assigned to node) | 
uct 
- Set per queue (also for users) 
- Enforced at each level in the 
hierarchy 
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QUEUE HIERARCHY & QUOTAS 


Example 


ene resource limits 
resource limit Ee / 5 CPU root Gd based on 
60 GB / 4 CPU registered nodes 
20GB / 2 CPU r E 30GB / 3 CPU 


10GB / 1 CPU 10GB / 1 CPU LOGE EUCEU 30GB / 3CPU 
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TRACKING RESOURCE USAGE 


How and what is tracked 


YUNIKORN 


- Allocation tracking as part of the scheduling cycle 
- Asynchronous updates for finished Allocations 


- Scheduler only cares about current state 


try m try Ai EN 
fail | 
finished «application 
allocation „node 
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TRACKING RESOURCE USAGE 


Deep dive into tracking data... 


YUNIKORN 


Quotas: 

- Usage tracked for quota enforcement: Current State 
7 Queue user 
ern group 


Hierarchical: enforced during scheduling cycles 
Application l application 
- Usage tracking only 

Node 

- Usage must always fit in available resources 
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HISTORICAL RESOURCE USAGE be 
Additional detail needed 


- YuniKorn is stateless 
D | € - re-init current state on startup 
SY - guarantees same state as previous run 
- Scheduler only cares about current state 


- Reusable identifiers 
- YuniKorn instance identification 


- Node is known by name only 
- Application ID may be reused 
- Configuration changes over time 


- Queues 
- Quotas 
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USAGE INFORMATION OVER TIME 


Event System 


Design guidelines 


Low memory consumption 
YuniKorn remains stateless 
No graphical user interface 
Existing events system 


Additional details 


States (application) 

Requests (application) 
Configuration changes (queues) 
Cluster changes (nodes) 
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Current State 


application 


requests 
a states 


cluster 
changes 
allocations 
current state 
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EVENT SYSTEM 


Usage information over time 


si.EventRecord { 
Type: 
ChangeType: 
ChangeDetail: 


TimestampNano: 


ObjectlD: 
ReferencelD: 
Resource: 
Message: 


CLOUDZRA 


eventRecordType, 
eventChangeType, 
eventChangeDetail, 
int64, 

String, 

String, 

si.Resource, 

String, 


REQUEST. ALLOC 


APP. ALLOC 
APP. REOUEST 
APP. REJECT 
APP. RUNNING 


NODE_ALLOC 
NODE_CAPACITY 


QUEUE_CONFIG 
QUEUE_MAX 
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REQUEST 


APP 
NODE 
QUEUE 


NONE 
SET 
ADD 
REMOVE 


// Request allocated 


e 
XAMples 


// Allocation changed 
// Request changed 
// Application rejected on create 
// State change to running 


// Allocation changed 
// Capacity changed 


// Managed queue update or removal 
// Max resource changed 
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EVENT SYSTEM 


Integration in the scheduling cycle EN 
configuration „W 
changes 


oe .- 


Ca 
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> O e 
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s 
try 


YUNIKORN 


reguests O O O „application 
& states 


| 
l 
l enode 
A I 
l l 
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pass 
— 


try m 
EB cC HT 


fail 
finished 
allocation 
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EVENT SYSTEM 


Limitations and possible future developments 
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Limitation vl, 
REST API: all or nothing b di 
No user or group events E gi 
Future 
Summarised usage 
Metrics collection replacement © 
Streaming support = 
Add on service == s: 
- Storage 
- Filtered events (REST) 


- De-duplicate events from restart 
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DEMO 


YUNIKORN 
Prepared cluster 


- Showing multiple pieces of functionality 
- Quotas 


- Preemption 
- EventSystem 


- Kind cluster (1.28.0) 
- Plugin version deployed 
- 3 nodes (control-plane + 2 workers) 
- Hierarchical queues, leafs defined: 
- Same quota (maximum usage) 
- Different guaranteed resources 
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Website: https://yunikorn.apache.org 


Email: dev(dyunikorn.apache.or 


Slack: YuniKorn Slack 
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